Efficient Inverse Reinforcement Learning using Adaptive State-Graphs

Inverse Reinforcement Learning (IRL) provides a powerful mechanism for learning complex behaviors from demonstration by rationalizing such demonstrations. Unfortunately its applicability has been largely hindered by lack of powerful representations that can take advantage of various task affordances while still admitting scalability. Inspired by the success of sampling based approaches in classical motion planning, we use adaptive state graphs to model the underlying Markov decision process (MDP) allowing us to further incorporate task specific constraints efficiently. We then develop a new Bayesian IRL (BIRL) algorithm to learn behaviors using sampled trajectories over the adaptive state graph. We demonstrate the effectiveness of this approach in the task of learning socially compliant robot navigation policies.